[vcpkg-tool] Replace curl command calls with libcurl #1660

vicroms · 2025-04-22T23:18:37Z

Summary by Copilot:

This PR replaces the existing curl command line tool usage with direct libcurl API calls throughout the vcpkg codebase. The main goal is to improve reliability, error handling, and reduce external dependencies by using libcurl directly instead of spawning curl subprocesses.

Install libcurl headers on Linux CI runners.
Replace curl command-line invocations with libcurl API calls
Add new libcurl infrastructure with proper initialization and error handling
Update test expectations to match new libcurl error message format
Make z-upload-metrics work on non-Windows platforms

Before merging these changes, we require confirmation from telemetry that the large majority of non-Windows users have a system-installed libcurl.

TODO:

Add license notice

cmake/FindLibCURL.cmake

src/vcpkg.cpp

src/vcpkg/base/downloads.cpp

Copilot

Pull Request Overview

This PR replaces the existing curl command line tool usage with direct libcurl API calls throughout the vcpkg codebase. The goal is to improve reliability, error handling, and reduce external dependencies by using libcurl directly instead of spawning curl subprocesses.

Key changes include:

Adding new libcurl infrastructure with proper initialization and error handling
Replacing all curl command-line invocations with libcurl API calls in download, upload, and metrics functionality
Making the z-upload-metrics command work on non-Windows platforms by removing the Windows-only restriction

Reviewed Changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/vcpkg/metrics.cpp	Replaces WinHTTP metrics upload with libcurl implementation and updates background process execution
src/vcpkg/commands.z-upload-metrics.cpp	Removes Windows-only restriction and adds file deletion after upload
src/vcpkg/base/downloads.cpp	Replaces all curl command-line calls with libcurl API calls for downloads, uploads, and HTTP requests
src/vcpkg/base/curl.cpp	Adds new libcurl global initialization infrastructure
include/vcpkg/base/curl.h	Provides libcurl header includes and initialization function
src/vcpkg/base/files.cpp	Adds file size methods to ReadFilePointer for libcurl upload functionality
CMakeLists.txt	Adds libcurl dependency and linking configuration
locales/messages.json	Updates error message formats to match libcurl error reporting

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/vcpkg/base/downloads.cpp

cmake/FindLibCURL.cmake

…ink-libcurl

…k-libcurl

src/vcpkg/base/curl.cpp

include/vcpkg/base/curl.h

src/vcpkg/base/curl.cpp

src/vcpkg/commands.z-upload-metrics.cpp

src/vcpkg/metrics.cpp

BillyONeal · 2025-11-10T21:25:28Z

src/vcpkg/metrics.cpp

+            maybe_accepted->is_integer() && maybe_errors->is_array())
+        {
+            auto item_received = maybe_received->integer(VCPKG_LINE_INFO);
+            auto item_accepted = maybe_accepted->integer(VCPKG_LINE_INFO);


Not a fan of this check followed by assert pattern but there's currently no maybe_integer so no change requested for now.

include/vcpkg/metrics.h

src/vcpkg/base/downloads.cpp

.github/workflows/build.yaml

cmake/FindLibCURL.cmake

BillyONeal · 2025-11-10T21:40:02Z

cmake/FindLibCURL.cmake

+    set(CMAKE_DISABLE_FIND_PACKAGE_ZLIB ON)
+    set(CMAKE_DISABLE_FIND_PACKAGE_LibPSL ON)
+    set(CMAKE_DISABLE_FIND_PACKAGE_LibSSH2 ON)
+    set(CMAKE_DISABLE_FIND_PACKAGE_Brotli ON)
+    set(CMAKE_DISABLE_FIND_PACKAGE_Zstd ON)


Another reason to consider hosting ourselves rather than FetchContent would be to more easily support ZLIB for Content-encoding: deflate or gzip.

Co-authored-by: Billy O'Neal <[email protected]> Co-authored-by: Thomas1664 <[email protected]>

cmake/FindLibCURL.cmake

src/vcpkg/base/downloads.cpp

For microsoft#1660

For #1660

…ft#1660)" This reverts commit 1cc117b. # Conflicts: # locales/messages.json

vicroms requested a review from Copilot August 21, 2025 23:42

This comment was marked as outdated.

Sign in to view

microsoft deleted a comment from Copilot AI Aug 22, 2025

vicroms force-pushed the link-libcurl branch 3 times, most recently from dedafaf to 8a25177 Compare August 22, 2025 09:37

talregev reviewed Aug 23, 2025

View reviewed changes

cmake/FindLibCURL.cmake Outdated Show resolved Hide resolved

vicroms commented Aug 23, 2025

View reviewed changes

cmake/FindLibCURL.cmake Outdated Show resolved Hide resolved

vicroms requested a review from Copilot August 24, 2025 21:44

This comment was marked as outdated.

Sign in to view

vicroms force-pushed the link-libcurl branch 2 times, most recently from 64cfadf to cddb978 Compare August 25, 2025 21:23

vicroms commented Aug 26, 2025

View reviewed changes

src/vcpkg.cpp Outdated Show resolved Hide resolved

src/vcpkg.cpp Outdated Show resolved Hide resolved

src/vcpkg/base/downloads.cpp Outdated Show resolved Hide resolved

Neumann-A and others added 2 commits August 28, 2025 12:26

add and link libcurl

d9cbe4a

Replace curl executable with libcurl API calls

d6aafe4

vicroms force-pushed the link-libcurl branch from 16bfe17 to d6aafe4 Compare August 28, 2025 19:38

microsoft deleted a comment from Copilot AI Aug 28, 2025

vicroms changed the title ~~[WIP][vcpkg-tool] Replace curl command calls with libcurl~~ [vcpkg-tool] Replace curl command calls with libcurl Aug 28, 2025

vicroms requested a review from Copilot August 28, 2025 22:26

Copilot AI reviewed Aug 28, 2025

View reviewed changes

src/vcpkg/base/downloads.cpp Outdated Show resolved Hide resolved

src/vcpkg/base/downloads.cpp Outdated Show resolved Hide resolved

xvitaly reviewed Sep 2, 2025

View reviewed changes

cmake/FindLibCURL.cmake Show resolved Hide resolved

vicroms added 6 commits September 2, 2025 21:04

Fix FindLibCURL for builds wihthout network access

f5a89c7

Merge branch 'main' of https://github.com/microsoft/vcpkg-tool into l…

5fc4d4e

…ink-libcurl

Merge branch 'main' of https://github.com/microsoft/vcpkg-tool into l…

7b40572

…ink-libcurl

Remove winHttp

e7f40e1

Restore progress function

da2529c

Handle non-dl transfer progress

2865022

vicroms added 10 commits November 5, 2025 15:27

Format and regenerate messages

622f98a

Merge branch 'main' of github.com:Microsoft/vcpkg-tool into link-libcurl

06cb1c3

Fix download retry

f7172f4

More review comments

2577dee

Handle response code from file:// downloads/

d76569b

Update end-to-end test expected output

5a2cd88

Fix download progress

c50352d

Add curl license and notices

e8f878d

Fix call to upload metrics

60dd34d

Fix expected output comparison with newer versions of libcurl

9cc1edc

vicroms marked this pull request as ready for review November 7, 2025 17:33

vicroms added 2 commits November 7, 2025 10:39

Fix expected output comparison with newer versions of libcurl

87b7169

Merge branch 'link-libcurl' of github.com:vicroms/vcpkg-tool into lin…

8e2018b

…k-libcurl

Thomas1664 reviewed Nov 9, 2025

View reviewed changes

BillyONeal previously approved these changes Nov 10, 2025

View reviewed changes

Apply suggestions from code review

1c53d79

Co-authored-by: Billy O'Neal <[email protected]> Co-authored-by: Thomas1664 <[email protected]>

vicroms dismissed BillyONeal’s stale review via 1c53d79 November 10, 2025 21:53

More PR comments

fd741eb

maksqwe reviewed Nov 11, 2025

View reviewed changes

cmake/FindLibCURL.cmake Show resolved Hide resolved

maksqwe reviewed Nov 11, 2025

View reviewed changes

src/vcpkg/base/downloads.cpp Outdated Show resolved Hide resolved

Disable curl examples and fix out-of-bounds access in retry loop

a9c5782

BillyONeal approved these changes Nov 11, 2025

View reviewed changes

vicroms merged commit 1cc117b into microsoft:main Nov 17, 2025
7 checks passed

BillyONeal mentioned this pull request Nov 17, 2025

[vcpkg tool] vcpkg does not implement Web Proxy Auto Detect on Windows microsoft/vcpkg#41987

Open

BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Nov 18, 2025

Add libcurl hooks for the signed build.

2dcba20

For microsoft#1660

BillyONeal mentioned this pull request Nov 18, 2025

Add libcurl hooks for the signed build. #1852

Merged

BillyONeal added a commit that referenced this pull request Nov 18, 2025

Add libcurl hooks for the signed build. (#1852)

fc48689

For #1660

BillyONeal mentioned this pull request Nov 18, 2025

Release vcpkg-tool 2025-11-19. microsoft/vcpkg#48303

Merged

BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Nov 18, 2025

Revert "[vcpkg-tool] Replace curl command calls with libcurl (microso…

64fa487

…ft#1660)" This reverts commit 1cc117b. # Conflicts: # locales/messages.json

BillyONeal mentioned this pull request Nov 18, 2025

Revert libcurl. #1856

Merged

[vcpkg-tool] Replace curl command calls with libcurl #1660

[vcpkg-tool] Replace curl command calls with libcurl #1660

Uh oh!

Conversation

vicroms commented Apr 22, 2025 • edited by BillyONeal Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BillyONeal Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BillyONeal Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

vicroms commented Apr 22, 2025 •

edited by BillyONeal

Loading